Jun 03 01:14:20.586045 np0000178251 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service. Jun 03 01:14:22.123567 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.common.config [-] Logging enabled! Jun 03 01:14:22.124165 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev352 Jun 03 01:14:22.124165 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=105636) setup_logging /opt/stack/neutron/neutron/common/config.py:125}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting... Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} Jun 03 01:14:22.125582 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [-] Loading core plugin: ml2 Jun 03 01:14:22.533647 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.533766 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.533806 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.534404 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.534539 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.534585 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.569575 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.569575 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.569778 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.576791 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.576893 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.576893 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.590772 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.590772 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.590937 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.628308 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.628479 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.628479 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.693201 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.693479 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.693557 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.693723 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.694291 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.756729 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.756922 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.756922 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.756922 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757126 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757126 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757126 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757365 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757365 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757365 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757365 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757666 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757666 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757666 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757666 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.757666 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758036 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758036 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758036 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758036 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758036 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758370 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758370 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758370 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.758644 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.758727 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.758727 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.758832 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.758832 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.758930 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:22.759101 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve'] Jun 03 01:14:22.759892 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public'] Jun 03 01:14:22.760566 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete Jun 03 01:14:22.761055 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]}) Jun 03 01:14:22.761135 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan'] Jun 03 01:14:22.761282 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan']) Jun 03 01:14:22.761353 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve'] Jun 03 01:14:22.761507 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos'] Jun 03 01:14:22.762101 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos'] Jun 03 01:14:22.762170 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos'] Jun 03 01:14:22.762250 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn'] Jun 03 01:14:22.802540 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn'] Jun 03 01:14:22.802682 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn'] Jun 03 01:14:22.802682 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling. Jun 03 01:14:22.803308 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.803427 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:22.823565 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' Jun 03 01:14:22.823634 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete Jun 03 01:14:22.823674 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' Jun 03 01:14:22.823809 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)] Jun 03 01:14:22.875307 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO dbcounter [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Registered counter for database neutron Jun 03 01:14:22.924315 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_db.sqlalchemy.engines [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105636) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}} Jun 03 01:14:23.360334 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG dbcounter [-] [105636] Writer thread running {{(pid=105636) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}} Jun 03 01:14:23.534930 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing driver for type 'local' Jun 03 01:14:23.535136 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing driver for type 'vlan' Jun 03 01:14:23.658369 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] VlanTypeDriver initialization complete Jun 03 01:14:23.658369 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing extension driver 'port_security' Jun 03 01:14:23.658631 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.extensions.port_security [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] PortSecurityExtensionDriver initialization complete Jun 03 01:14:23.658631 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing extension driver 'qos' Jun 03 01:14:23.658631 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.plugins.ml2.extensions.qos [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QosExtensionDriver initialization complete {{(pid=105636) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}} Jun 03 01:14:23.658808 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing mechanism driver 'ovn' Jun 03 01:14:23.658808 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Starting OVNMechanismDriver Jun 03 01:14:23.660622 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > process before_spawn 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.660753 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > process after_init 55550000, True {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.660832 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > segment after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.660904 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > segment after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.660978 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > segment after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661102 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > allowed_address_pair before_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661173 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661278 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661388 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661479 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group precommit_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661541 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661616 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661717 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661803 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661866 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661927 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.661986 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.662185 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > qos_plugin after_init 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.662373 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > trunk_plugin after_init 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.662615 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > logging-plugin after_init 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.663173 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.services.logapi.drivers.ovn.driver [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVN logging driver registered Jun 03 01:14:23.663295 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: ._register_pvlan_driver at 0x7c7999eea3e0> pvlan_plugin after_init 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.664623 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.plugins.ml2.plugin [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=105636) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}} Jun 03 01:14:23.664873 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.664956 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665020 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665135 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > subnet after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665225 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > subnet after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665305 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > subnet after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665369 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > network after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665429 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > network after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665542 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > network after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665626 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665703 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665766 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665825 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.665886 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.666000 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > security_group_rule after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.666113 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.666113 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.666231 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > address_group after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.666293 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ML2 OVO RPC backend initialized. {{(pid=105636) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}} Jun 03 01:14:23.667151 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.quota [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded quota_driver: . Jun 03 01:14:23.667242 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.plugin [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Modular L2 Plugin initialization complete Jun 03 01:14:23.667344 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading services supported by the core plugin {{(pid=105636) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}} Jun 03 01:14:23.667454 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.plugins.ml2.managers [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Got port-security extension from driver 'port_security' Jun 03 01:14:23.667716 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=105636) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}} Jun 03 01:14:23.667781 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: ovn-router Jun 03 01:14:23.676661 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.676769 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: port before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.676831 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: port after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.676892 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port precommit_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677039 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: port before_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677149 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: port before_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677149 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677270 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677330 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677392 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: subnet after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.677610 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Tracking information for resource: router configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:23.677700 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.quota.resource_registry [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Tracking information for resource: floatingip configured {{(pid=105636) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}} Jun 03 01:14:23.677795 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.services.ovn_l3.plugin [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Starting OVNL3RouterPlugin Jun 03 01:14:23.677931 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port_forwarding_plugin after_init 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.678111 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router before_create 55550100, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.678178 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_delete 55550100, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.678270 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_create 55550100, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.678360 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_update 55550100, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.681162 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.services.provider_configuration [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Service providers = [] {{(pid=105636) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}} Jun 03 01:14:23.681284 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.db.servicetype_db [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=105636) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}} Jun 03 01:14:23.682586 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_create 55549900, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.682691 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router_interface after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.682781 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.682847 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.682923 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip precommit_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683003 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683080 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip after_status_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683211 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > floatingip after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683304 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > port_forwarding after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683369 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router_interface after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683438 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router after_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683574 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683659 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router after_delete 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683659 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > router after_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683772 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.services.service_base [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=105636) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} Jun 03 01:14:23.683859 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: > process after_init 55550001, True {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.683997 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.684096 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: auto_allocate Jun 03 01:14:23.684809 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: network precommit_update 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.684893 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron_lib.callbacks.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Subscribe: network precommit_create 55550000, False {{(pid=105636) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:23.685081 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.685153 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: tag Jun 03 01:14:23.687456 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.687519 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: timestamp Jun 03 01:14:23.688679 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.688851 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: network_ip_availability Jun 03 01:14:23.690153 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.690222 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: flavors Jun 03 01:14:23.691138 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.691138 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading Plugin: revisions Jun 03 01:14:23.692006 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.manager [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=105636) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}} Jun 03 01:14:23.692121 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_concurrency.lockutils [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.567s {{(pid=105636) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} Jun 03 01:14:23.692316 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=105636) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}} Jun 03 01:14:23.692392 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Initializing extension manager. Jun 03 01:14:23.692658 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: __init__.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.692762 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: __pycache__ {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.692824 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: aap_reject_multicast.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.693045 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Allowed address pairs reject multicast extension" alias="aap-reject-multicast" description="Reject multicast addresses in Allowed Address Pairs" updated="2026-04-01T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.693253 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension aap-reject-multicast not supported by any of loaded plugins Jun 03 01:14:23.693322 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: address_group.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.693631 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.693789 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: address-group Jun 03 01:14:23.693891 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: address_scope.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.694110 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.694245 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: address-scope Jun 03 01:14:23.694308 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: admin_state_down_before_update.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.694537 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.694672 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins Jun 03 01:14:23.694736 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: agent.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.694991 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.695130 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: agent Jun 03 01:14:23.695253 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: agent_resources_synced.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.695467 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.695623 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension agent-resources-synced not supported by any of loaded plugins Jun 03 01:14:23.695697 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: agent_sort_key.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.695888 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.696025 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension agent_sort_key not supported by any of loaded plugins Jun 03 01:14:23.696091 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: allowedaddresspairs.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.696661 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.696828 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: allowed-address-pairs Jun 03 01:14:23.696914 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: auto_allocated_topology.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.697110 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.697228 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: auto-allocated-topology Jun 03 01:14:23.697305 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: availability_zone.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.697577 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.697719 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: availability_zone Jun 03 01:14:23.697801 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: availability_zone_filter.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.697979 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.698146 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension availability_zone_filter not supported by any of loaded plugins Jun 03 01:14:23.698237 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: data_plane_status.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.698400 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.698600 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension data-plane-status not supported by any of loaded plugins Jun 03 01:14:23.698672 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: default_subnetpools.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.698886 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.698990 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: default-subnetpools Jun 03 01:14:23.699068 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: dhcpagentscheduler.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.699427 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.699587 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: dhcp_agent_scheduler Jun 03 01:14:23.699659 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: dns.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.699844 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.699989 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: dns-integration Jun 03 01:14:23.700070 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: dns_domain_ports.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.700257 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.700390 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: dns-domain-ports Jun 03 01:14:23.700481 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.700674 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like , , and as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.700804 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: dns-integration-domain-keywords Jun 03 01:14:23.700888 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: dvr.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.701095 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.701230 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension dvr not supported by any of loaded plugins Jun 03 01:14:23.701316 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: ecmp_routes.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.701517 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.701653 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension ecmp_routes not supported by any of loaded plugins Jun 03 01:14:23.701724 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: empty_string_filtering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.701921 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.702059 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: empty-string-filtering Jun 03 01:14:23.702131 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: evpn.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.702322 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="EVPN" alias="evpn" description="Router extension to support EVPN Type-5 route advertisement" updated="2026-04-28T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.702473 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension evpn not supported by any of loaded plugins Jun 03 01:14:23.702550 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.702741 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.702876 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins Jun 03 01:14:23.702948 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.703140 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.703329 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins Jun 03 01:14:23.703402 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: external_net.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.703627 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.703754 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: external-net Jun 03 01:14:23.703824 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: extra_dhcp_opt.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.704019 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.704155 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: extra_dhcp_opt Jun 03 01:14:23.704223 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: extraroute.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.704416 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.704566 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: extraroute Jun 03 01:14:23.704636 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: extraroute_atomic.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.704841 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.705003 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: extraroute-atomic Jun 03 01:14:23.705049 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: filter_validation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.705257 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.705414 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: filter-validation Jun 03 01:14:23.705512 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: fip_pf_description.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.705734 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.705871 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins Jun 03 01:14:23.705937 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: fip_pf_detail.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.706269 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.706412 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: floating-ip-port-forwarding-detail Jun 03 01:14:23.706515 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: fip_pf_port_range.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.706711 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.706877 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins Jun 03 01:14:23.706946 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: fip_port_details.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.707154 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.707310 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: fip-port-details Jun 03 01:14:23.707391 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: flavors.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.707628 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.707776 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: flavors Jun 03 01:14:23.707848 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.708127 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.708265 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins Jun 03 01:14:23.708337 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: floatingip_pools.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.708575 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.708723 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: floatingip-pools Jun 03 01:14:23.708789 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: ip_allocation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.708976 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.709117 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension ip_allocation not supported by any of loaded plugins Jun 03 01:14:23.709188 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.709376 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.709527 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension ip-substring-filtering not supported by any of loaded plugins Jun 03 01:14:23.709610 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l2_adjacency.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.709799 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.709932 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l2_adjacency not supported by any of loaded plugins Jun 03 01:14:23.710015 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.710366 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.710539 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: router Jun 03 01:14:23.710615 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_conntrack_helper.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.711117 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.711262 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l3-conntrack-helper not supported by any of loaded plugins Jun 03 01:14:23.711426 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.711636 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.711788 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: enable-default-route-bfd Jun 03 01:14:23.711843 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.712048 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.712189 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: enable-default-route-ecmp Jun 03 01:14:23.712260 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.712457 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.712604 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: ext-gw-mode Jun 03 01:14:23.712676 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.712860 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.712992 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: l3-ha Jun 03 01:14:23.713064 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.713251 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.713386 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins Jun 03 01:14:23.713471 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_extra_gws.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.713650 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.713784 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: external-gateway-multihoming Jun 03 01:14:23.713860 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_flavors.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.714046 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.714177 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: l3-flavors Jun 03 01:14:23.714248 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_ndp_proxy.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.714514 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.714667 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l3-ndp-proxy not supported by any of loaded plugins Jun 03 01:14:23.714744 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.714923 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.715056 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins Jun 03 01:14:23.715122 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: l3agentscheduler.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.715612 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.715742 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension l3_agent_scheduler not supported by any of loaded plugins Jun 03 01:14:23.715809 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: local_ip.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.716094 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.716254 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension local_ip not supported by any of loaded plugins Jun 03 01:14:23.716355 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: logging.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.716599 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.716726 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension logging not supported by any of loaded plugins Jun 03 01:14:23.716795 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: metering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.717182 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.717318 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension metering not supported by any of loaded plugins Jun 03 01:14:23.717377 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.717616 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.717747 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins Jun 03 01:14:23.717795 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: multiprovidernet.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.718010 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.718142 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: multi-provider Jun 03 01:14:23.718195 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: netmtu.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.718390 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.718541 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: net-mtu Jun 03 01:14:23.718611 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: netmtu_writable.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.718794 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.718923 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: net-mtu-writable Jun 03 01:14:23.718993 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: network_availability_zone.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.719192 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.719373 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: network_availability_zone Jun 03 01:14:23.719426 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: network_ha.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.719635 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.719763 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension network_ha not supported by any of loaded plugins Jun 03 01:14:23.719834 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: network_ip_availability.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.720032 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.720160 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: network-ip-availability Jun 03 01:14:23.720228 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: network_ip_availability_details.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.720433 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.720584 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: network-ip-availability-details Jun 03 01:14:23.720654 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: network_segment_range.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.720909 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.721040 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension network-segment-range not supported by any of loaded plugins Jun 03 01:14:23.721108 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: pagination.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.721286 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.721350 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: pagination Jun 03 01:14:23.721456 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_device_profile.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.721634 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.721767 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: port-device-profile Jun 03 01:14:23.721827 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_hardware_offload_type.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.722020 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.722149 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-hardware-offload-type not supported by any of loaded plugins Jun 03 01:14:23.722211 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.722387 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.722540 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins Jun 03 01:14:23.722601 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_hints.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.722784 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.722910 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-hints not supported by any of loaded plugins Jun 03 01:14:23.722991 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_mac_address_override.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.723165 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.723293 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-mac-override not supported by any of loaded plugins Jun 03 01:14:23.723593 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.723593 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.723697 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: port-mac-address-regenerate Jun 03 01:14:23.723774 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.723948 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.724087 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: port-numa-affinity-policy Jun 03 01:14:23.724153 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.724325 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.724488 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: port-numa-affinity-policy-socket Jun 03 01:14:23.724708 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_resource_request.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.724708 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.724865 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-resource-request not supported by any of loaded plugins Jun 03 01:14:23.724926 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_resource_request_groups.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.725130 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.725250 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-resource-request-groups not supported by any of loaded plugins Jun 03 01:14:23.725318 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: port_trusted_vif.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.725549 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.725646 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-trusted-vif not supported by any of loaded plugins Jun 03 01:14:23.725715 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: portbindings.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.725892 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.726030 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: binding Jun 03 01:14:23.726098 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: portbindings_extended.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.726315 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.726452 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: binding-extended Jun 03 01:14:23.726514 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: portsecurity.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.726712 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.726839 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: port-security Jun 03 01:14:23.726907 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: project_id.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.727098 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.727175 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: project-id Jun 03 01:14:23.727240 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: providernet.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.727518 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.727602 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: provider Jun 03 01:14:23.727692 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: pvlan.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.727881 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Private VLAN" alias="pvlan" description="The Private VLAN semantics extension." updated="2026-02-03T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.727995 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension pvlan not supported by any of loaded plugins Jun 03 01:14:23.728065 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qinq.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.728256 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x88a8 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.728417 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: qinq Jun 03 01:14:23.728479 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.729241 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.729369 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos not supported by any of loaded plugins Jun 03 01:14:23.729424 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.729689 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.729821 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins Jun 03 01:14:23.729894 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.730100 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.730228 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins Jun 03 01:14:23.730285 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_default.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.730522 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.730661 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-default not supported by any of loaded plugins Jun 03 01:14:23.730721 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_fip.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.730914 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.731057 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-fip not supported by any of loaded plugins Jun 03 01:14:23.731128 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_fip_network_policy.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.731311 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.731495 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-fip-network-policy not supported by any of loaded plugins Jun 03 01:14:23.731570 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_gateway_ip.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.731759 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.731891 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-gateway-ip not supported by any of loaded plugins Jun 03 01:14:23.731957 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_port_network_policy.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.732150 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.732281 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-port-network-policy not supported by any of loaded plugins Jun 03 01:14:23.732348 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.732568 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.732699 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-pps-minimum not supported by any of loaded plugins Jun 03 01:14:23.732771 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.732961 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.733104 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins Jun 03 01:14:23.733177 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_pps_rule.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.733374 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.733537 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-pps not supported by any of loaded plugins Jun 03 01:14:23.733610 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_rule_type_details.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.733786 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.733934 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-rule-type-details not supported by any of loaded plugins Jun 03 01:14:23.734004 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_rule_type_filter.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.734194 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.734330 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-rule-type-filter not supported by any of loaded plugins Jun 03 01:14:23.734403 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: qos_rules_alias.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.734635 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.734759 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension qos-rules-alias not supported by any of loaded plugins Jun 03 01:14:23.734831 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: quota_check_limit.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.735048 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.735166 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: quota-check-limit Jun 03 01:14:23.735240 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: quota_check_limit_default.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.735484 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.735622 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: quota-check-limit-default Jun 03 01:14:23.735691 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: quotasv2.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.736095 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.736171 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: quotas Jun 03 01:14:23.736237 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: quotasv2_detail.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.736692 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.736776 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: quota_details Jun 03 01:14:23.736865 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: rbac.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.737265 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.737410 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: rbac-policies Jun 03 01:14:23.737498 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: rbac_address_group.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.737701 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.737829 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension rbac-address-group not supported by any of loaded plugins Jun 03 01:14:23.737919 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: rbac_address_scope.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.738103 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.738263 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: rbac-address-scope Jun 03 01:14:23.738328 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: rbac_security_groups.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.738543 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.738685 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: rbac-security-groups Jun 03 01:14:23.738749 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: rbac_subnetpool.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.738943 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.739072 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension rbac-subnetpool not supported by any of loaded plugins Jun 03 01:14:23.739145 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: revisionifmatch.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.739342 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.739523 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: revision-if-match Jun 03 01:14:23.739592 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: revisions.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.739795 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.739925 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: standard-attr-revisions Jun 03 01:14:23.739997 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: router_availability_zone.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.740203 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.740337 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: router_availability_zone Jun 03 01:14:23.740410 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: router_enable_snat.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.740616 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.740749 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: router-enable-snat Jun 03 01:14:23.740820 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: routerservicetype.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.741001 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.741134 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension router-service-type not supported by any of loaded plugins Jun 03 01:14:23.741205 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_default_rules.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.741581 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.741724 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-groups-default-rules Jun 03 01:14:23.741774 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.741972 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.742111 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-groups-normalized-cidr Jun 03 01:14:23.742176 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_port_filtering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.742365 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.742520 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension port-security-groups-filtering not supported by any of loaded plugins Jun 03 01:14:23.742589 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.742769 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.742901 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-groups-remote-address-group Jun 03 01:14:23.742970 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.743152 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.743284 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-groups-rules-belongs-to-default-sg Jun 03 01:14:23.743357 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.743635 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.743771 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-groups-shared-filtering Jun 03 01:14:23.743830 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: securitygroup.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.745265 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.745414 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: security-group Jun 03 01:14:23.745518 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: segment.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.745771 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.745922 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension segment not supported by any of loaded plugins Jun 03 01:14:23.745988 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.746192 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.746324 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins Jun 03 01:14:23.746395 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: servicetype.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.746657 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.746790 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: service-type Jun 03 01:14:23.746890 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: sorting.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.747077 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.747161 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: sorting Jun 03 01:14:23.747232 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: standard_attr_segment.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.747420 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.747607 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension standard-attr-segment not supported by any of loaded plugins Jun 03 01:14:23.747678 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: standardattrdescription.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.747872 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.748006 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: standard-attr-description Jun 03 01:14:23.748092 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: stateful_security_group.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.748283 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.748419 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: stateful-security-group Jun 03 01:14:23.748528 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: stdattrs_common.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.748696 np0000178251 neutron-ovn-maintenance-worker[105636]: WARNING neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py Jun 03 01:14:23.748767 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.748950 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.749083 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnet-dns-publish-fixed-ip Jun 03 01:14:23.749167 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnet_external_network.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.749347 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.749511 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnet-external-network Jun 03 01:14:23.749592 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnet_onboard.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.749801 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.749935 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnet_onboard Jun 03 01:14:23.750004 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.750193 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.750324 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins Jun 03 01:14:23.750395 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnet_service_types.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.750593 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.750724 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnet-service-types Jun 03 01:14:23.750808 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnetallocation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.751117 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.751286 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnet_allocation Jun 03 01:14:23.751383 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.751700 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.751897 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: subnetpool-prefix-ops Jun 03 01:14:23.751992 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: tag_creation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.752225 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.752395 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: tag-creation Jun 03 01:14:23.752516 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.752774 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.752946 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins Jun 03 01:14:23.753042 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: tagging.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.753745 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.753892 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: standard-attr-tag Jun 03 01:14:23.753950 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: timestamp.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.754173 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.754312 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: standard-attr-timestamp Jun 03 01:14:23.754378 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: trunk.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.754638 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.754769 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension trunk not supported by any of loaded plugins Jun 03 01:14:23.754841 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: trunk_details.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.755037 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.755162 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension trunk-details not supported by any of loaded plugins Jun 03 01:14:23.755325 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: uplink_status_propagation.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.755564 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.755694 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension uplink-status-propagation not supported by any of loaded plugins Jun 03 01:14:23.755764 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.755950 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.756080 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins Jun 03 01:14:23.756138 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loading extension file: vlantransparent.py {{(pid=105636) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}} Jun 03 01:14:23.756339 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=105636) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}} Jun 03 01:14:23.756498 np0000178251 neutron-ovn-maintenance-worker[105636]: INFO neutron.api.extensions [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Loaded extension: vlan-transparent Jun 03 01:14:23.761636 np0000178251 neutron-ovn-maintenance-worker[105636]: WARNING oslo_service.backend._threading.service [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback. Jun 03 01:14:23.761957 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Full set of CONF: {{(pid=105636) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}} Jun 03 01:14:23.762020 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ******************************************************************************** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}} Jun 03 01:14:23.762089 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Configuration options gathered from: {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}} Jun 03 01:14:23.762159 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}} Jun 03 01:14:23.762259 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}} Jun 03 01:14:23.762325 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ================================================================================ {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}} Jun 03 01:14:23.762451 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] agent_down_time = 75 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.762545 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] allow_automatic_dhcp_failover = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.762631 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] allow_bulk = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.762807 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.762891 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] api_extensions_path = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.762980 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] api_paste_config = api-paste.ini {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763084 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] api_workers = 4 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763169 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] auth_strategy = keystone {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763255 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] base_mac = fa:16:3e:00:00:00 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763363 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] bind_host = 0.0.0.0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763476 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] bind_port = 9696 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763611 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] client_socket_timeout = 900 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763704 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] config_dir = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763769 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763878 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] config_source = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.763944 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] control_exchange = neutron {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764034 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] core_plugin = ml2 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764165 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] debug = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764253 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] default_availability_zones = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764409 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] default_log_levels = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764513 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dhcp_agent_notification = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764599 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dhcp_agents_per_network = 1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764683 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dhcp_lease_duration = 86400 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764772 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dhcp_load_type = networks {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764859 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dns_domain = openstacklocal {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.764947 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765032 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_default_route_bfd = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765116 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_default_route_ecmp = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765199 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_dvr = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765281 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_new_agents = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765369 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765463 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_snat_by_default = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765567 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] enable_traditional_dhcp = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765649 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] external_dns_driver = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765731 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] filter_validation = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765821 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] global_physnet_mtu = 1430 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.765883 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] graceful_shutdown_timeout = 60 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766022 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] host = np0000178251 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766108 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] host_dvr_for_dhcp = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766192 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] http_retries = 3 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766281 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] instance_format = [instance: %(uuid)s] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766367 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766482 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ipam_driver = internal {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766562 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ipv6_pd_enabled = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766646 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] l3_ha = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766736 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766821 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] l3_ha_network_physical_name = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766907 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] l3_ha_network_type = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.766997 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_color = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767082 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_config_append = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767172 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767255 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_dir = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767340 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_file = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767403 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_options = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767513 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_rotate_interval = 1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767649 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_rotate_interval_type = days {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767732 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] log_rotation_type = none {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767797 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767861 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.767951 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768042 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768107 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768191 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_allowed_address_pair = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768280 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_dns_nameservers = 5 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768386 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_header_line = 16384 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768525 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_l3_agents_per_router = 3 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768607 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_logfile_count = 30 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768692 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_logfile_size_mb = 200 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768780 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_routes = 30 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768861 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] max_subnet_host_routes = 20 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.768949 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] my_ip = 199.19.213.61 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769039 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] my_ipv6 = 2604:e100:1:0:f816:3eff:fe3f:9939 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769123 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_auto_schedule = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769204 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_link_prefix = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769296 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769390 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] notify_nova_on_port_data_changes = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769551 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] notify_nova_on_port_status_changes = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769663 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] pagination_max_limit = -1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769754 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] periodic_fuzzy_delay = 5 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769838 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] periodic_interval = 40 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.769924 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] publish_errors = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770007 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rate_limit_burst = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770097 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rate_limit_except_level = CRITICAL {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770179 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rate_limit_interval = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770264 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] router_distributed = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770377 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rpc_resources_processing_step = 20 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770426 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rpc_response_max_timeout = 600 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770539 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rpc_response_timeout = 60 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770631 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rpc_state_report_workers = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770717 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] rpc_workers = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770793 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] send_events_interval = 2 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770888 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] service_plugins = ['ovn-router'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.770977 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] setproctitle = on {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771063 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] shell_completion = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771151 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] state_path = /opt/stack/data/neutron {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771239 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] syslog_log_facility = LOG_USER {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771325 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] tcp_keepidle = 600 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771413 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] transport_url = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771515 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] use_journal = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771640 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] use_json = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771714 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] use_stderr = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771821 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] use_syslog = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771892 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] vlan_qinq = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.771985 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] vlan_transparent = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.772084 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] wsgi_default_pool_size = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.772154 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] wsgi_keep_alive = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.772234 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.772320 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] wsgi_server_debug = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.772427 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_concurrency.disable_process_locking = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.772546 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.772634 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.connection_string = messaging:// {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.772731 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.enabled = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.772825 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.es_doc_type = notification {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.772915 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.es_scroll_size = 10000 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773004 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.es_scroll_time = 2m {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773091 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.filter_error_trace = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773182 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.hmac_keys = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773277 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.sentinel_service_name = mymaster {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773375 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.socket_timeout = 0.1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773482 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.trace_requests = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773574 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler.trace_sqlalchemy = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773670 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler_jaeger.process_tags = {} {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773755 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler_jaeger.service_name_prefix = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773842 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] profiler_otlp.service_name_prefix = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.773938 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774028 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774118 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774210 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774299 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774401 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.enforce_new_defaults = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774601 np0000178251 neutron-ovn-maintenance-worker[105636]: WARNING oslo_config.cfg [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.). Its value may be silently ignored in the future. Jun 03 01:14:23.774703 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.enforce_scope = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774793 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.policy_default_rule = default {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774896 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.774987 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775084 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775168 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775256 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775342 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775429 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775537 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_policy.remote_timeout = 60.0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775699 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775786 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] service_providers.service_provider = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775881 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.auth_section = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.775978 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.auth_type = password {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776073 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.auth_url = https://199.19.213.61/identity {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776161 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.cafile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776241 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.certfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776343 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.collect_timing = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776451 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.default_domain_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776551 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.default_domain_name = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776635 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.domain_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776719 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.domain_name = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776814 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.endpoint_type = public {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776897 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.insecure = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.776983 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.keyfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777075 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.password = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777163 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.project_domain_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777254 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.project_domain_name = Default {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777342 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.project_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777452 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.project_name = service {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777547 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.region_name = RegionOne {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777636 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.split_loggers = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777720 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.system_scope = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777803 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.timeout = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777888 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.trust_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.777973 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.user_domain_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778065 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.user_domain_name = Default {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778148 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.user_id = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778243 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] nova.username = nova {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778336 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.auth_section = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778428 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.auth_type = password {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778532 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.cafile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778619 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.certfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778706 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.collect_timing = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778797 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.endpoint_type = public {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778882 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.insecure = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.778968 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.keyfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779050 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.region_name = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779135 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.split_loggers = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779217 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] placement.timeout = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779309 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.auth_section = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779396 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.auth_type = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779496 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.cafile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779580 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.certfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779713 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.collect_timing = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779800 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.connect_retries = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779885 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.connect_retry_delay = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.779973 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.enable_notifications = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780056 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.endpoint_override = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780143 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.insecure = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780226 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.interface = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780306 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.keyfile = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780388 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.max_version = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780489 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.min_version = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780574 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.region_name = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780657 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.retriable_status_codes = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780741 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.service_name = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780824 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.service_type = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780908 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.split_loggers = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.780994 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.status_code_retries = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781076 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.status_code_retry_delay = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781158 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.timeout = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781243 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.valid_interfaces = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781324 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ironic.version = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781412 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] cli_script.dry_run = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781531 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781617 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781704 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781792 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781877 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.781965 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782053 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782137 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782229 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.hostname = np0000178251 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782316 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782410 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782521 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782610 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782699 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782791 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782878 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.782986 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783079 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783165 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783252 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783346 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783429 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783536 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783625 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783772 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783861 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.783947 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784031 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784114 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784214 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784317 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784424 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784537 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784624 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784712 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784800 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.ssl_version = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784885 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.784978 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_notifications.driver = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785074 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_notifications.retry = -1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785170 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785254 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_messaging_notifications.transport_url = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785343 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_reports.file_event_handler = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785427 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785527 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_reports.log_dir = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785619 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785722 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.default_quota = -1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785818 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_driver = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785905 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_floatingip = 50 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.785991 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_network = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786076 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_port = 500 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786183 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_rbac_policy = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786260 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_router = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786352 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_security_group = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786436 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786545 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.quota_subnet = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786647 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] QUOTAS.track_quota_usage = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786748 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786831 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.786915 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787001 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787092 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787194 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787278 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787370 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787458 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787554 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787638 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787814 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.787917 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788003 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_dhcp_release.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788094 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788176 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788260 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788364 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788492 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788579 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788659 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_ovs_vsctl.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788757 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.capabilities = [21] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788839 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.788920 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789007 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789094 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789177 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789260 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_namespace.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789349 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.capabilities = [12] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789430 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789530 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789619 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789706 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789788 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789867 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_conntrack.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.789956 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.capabilities = [12, 21] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790040 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.group = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790123 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.helper_command = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790207 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.log_daemon_traceback = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790294 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.logger_name = oslo_privsep.daemon {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790376 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.thread_pool_size = 16 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790474 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] privsep_link.user = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790573 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] AGENT.root_helper = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790667 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] AGENT.root_helper_daemon = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790753 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] AGENT.use_helper_for_ns_read = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790854 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] SECURITYGROUP.enable_ipset = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.790936 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] SECURITYGROUP.enable_security_group = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791019 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] SECURITYGROUP.firewall_driver = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791109 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791198 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] experimental.ipv6_pd_enabled = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791298 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.extension_drivers = ['port_security', 'qos'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791382 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.external_network_type = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791494 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.mechanism_drivers = ['ovn'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791588 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.overlay_ip_version = 4 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791671 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.path_mtu = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791791 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.physical_network_mtus = {} {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791889 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.project_network_types = ['geneve'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.791991 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.tunnelled_network_rp_name = rp_tunnelled {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792089 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2.type_drivers = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792190 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2_type_flat.flat_networks = ['public'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792286 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2_type_geneve.max_header_size = 38 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792387 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2_type_geneve.vni_ranges = ['1:65536'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792505 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792591 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_log.burst_limit = 25 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792679 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_log.local_output_log_base = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792764 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] network_log.rate_limit = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792857 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.asyncio_connection = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.792990 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.asyncio_slave_connection = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793021 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.backend = sqlalchemy {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793122 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.connection = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793205 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.connection_debug = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793293 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.connection_parameters = {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793376 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.connection_recycle_time = 3600 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793476 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.connection_trace = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793566 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.db_inc_retry_interval = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793647 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.db_max_retries = 20 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793737 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.db_max_retry_interval = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793821 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.db_retry_interval = 1 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793903 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.max_overflow = 50 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.793986 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.max_pool_size = 5 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794072 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.max_retries = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794163 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794254 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.mysql_wsrep_sync_wait = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794337 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.pool_timeout = None {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794423 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.retry_interval = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794525 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.slave_connection = **** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794611 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.sqlite_synchronous = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794695 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.synchronous_reader = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794778 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] database.use_db_reconnect = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794880 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.bfd_min_rx = 1000 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.794965 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.bfd_min_tx = 100 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795050 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.bfd_mult = 3 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795136 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795221 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.dhcp_default_lease_time = 43200 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795310 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795393 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.dns_records_ovn_owned = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795498 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.dns_servers = [] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795586 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.enable_distributed_floating_ip = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795678 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.fdb_age_threshold = 300 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795785 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ha_failover_strategy = normal {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795873 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.live_migration_activation_strategy = rarp {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.795958 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.localnet_learn_fdb = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796049 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.mac_binding_age_threshold = 300 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796142 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.neutron_sync_mode = log {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796228 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_dhcp4_global_options = {} {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796324 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_dhcp6_global_options = {} {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796403 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_l3_scheduler = leastloaded {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796513 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_metadata_enabled = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796606 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_nb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796694 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_nb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796787 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_nb_connection = ['ssl:199.19.213.61:6641'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796876 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_nb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.796962 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_router_indirect_snat = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797052 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_sb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797185 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_sb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797234 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_sb_connection = ['ssl:199.19.213.61:6642'] {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797327 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovn_sb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797409 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovs_create_tap = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797520 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovsdb_connection_timeout = 180 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797621 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovsdb_log_level = INFO {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797688 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovsdb_probe_interval = 60000 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797773 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.ovsdb_retry_max_interval = 180 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797864 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.stateless_nat_enabled = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.797943 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn.vhost_sock_dir = /var/run/openvswitch {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798032 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.bridge_mac_table_size = 50000 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798120 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.igmp_flood = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798209 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.igmp_flood_reports = True {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798313 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.igmp_flood_unregistered = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798376 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.igmp_snooping_enable = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798495 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] OVS.ovsdb_timeout = 10 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798587 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798745 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn_nb_global.ignore_lsp_down = False {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798745 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.798795 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG cotyledon.oslo_config_glue [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] ******************************************************************************** {{(pid=105636) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}} Jun 03 01:14:23.798880 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG oslo_service.backend._threading.service [None req-6acf1d53-c1b4-425a-ae4d-c71da415dd56 None None] Selected the multiprocessing context: and updated it in Cotyledon ServiceManager: {{(pid=105636) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:405}} Jun 03 01:14:23.829120 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=106388) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}} Jun 03 01:14:23.829693 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3229}} Jun 03 01:14:23.829693 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3230}} Jun 03 01:14:23.829693 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3231}} Jun 03 01:14:23.829693 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3232}} Jun 03 01:14:23.830079 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3237}} Jun 03 01:14:23.830079 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time = 75 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.830237 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.830420 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.830760 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.830889 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831000 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config = api-paste.ini {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831185 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] api_workers = 4 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831262 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy = keystone {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831411 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] base_mac = fa:16:3e:00:00:00 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831758 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] bind_host = 0.0.0.0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.831972 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] bind_port = 9696 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832066 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout = 900 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832242 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] config_dir = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832302 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832422 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] config_source = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832571 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] control_exchange = neutron {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] core_plugin = ml2 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832819 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] debug = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.832939 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.833210 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.833332 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.833659 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network = 1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.833765 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration = 86400 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.833894 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type = networks {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834013 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dns_domain = openstacklocal {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834145 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834261 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834390 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834657 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834782 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.834891 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835004 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835098 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835221 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835280 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835382 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] filter_validation = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835544 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu = 1430 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.835635 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout = 60 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.836058 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] host = np0000178251 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837351 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837429 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] http_retries = 3 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837531 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] instance_format = [instance: %(uuid)s] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837612 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837670 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver = internal {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837768 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837849 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] l3_ha = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.837958 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838041 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838112 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838180 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_color = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838264 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_config_append = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838354 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838421 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_dir = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838496 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_file = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838549 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_options = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838625 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval = 1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838702 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type = days {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838788 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type = none {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838872 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838928 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.838970 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839029 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839068 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839141 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839219 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers = 5 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839316 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_header_line = 16384 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839399 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router = 3 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839493 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count = 30 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839562 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb = 200 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839637 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_routes = 30 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839701 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes = 20 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839786 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] my_ip = 199.19.213.61 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839875 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6 = 2604:e100:1:0:f816:3eff:fe3f:9939 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.839939 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840027 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840123 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840235 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840330 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840416 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit = -1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840513 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay = 5 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840578 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval = 40 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840643 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] publish_errors = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840693 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840777 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level = CRITICAL {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840839 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840989 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] router_distributed = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.840989 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step = 20 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841091 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout = 600 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841162 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout = 60 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841272 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841358 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841427 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval = 2 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841537 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] service_plugins = ['ovn-router'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841623 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] setproctitle = on {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841714 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] shell_completion = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841792 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] state_path = /opt/stack/data/neutron {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841862 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility = LOG_USER {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.841926 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle = 600 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842023 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] transport_url = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842102 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] use_journal = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842155 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] use_json = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842199 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] use_stderr = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842290 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] use_syslog = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842351 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842406 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842502 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842560 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842639 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842704 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3245}} Jun 03 01:14:23.842827 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.842927 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string = messaging:// {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843147 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843233 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type = notification {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843320 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size = 10000 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843406 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time = 2m {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843503 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843588 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843659 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843761 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout = 0.1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843825 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.843902 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844030 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags = {} {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844106 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844184 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844287 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844379 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844488 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844573 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844635 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844739 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844789 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844845 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844889 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.844968 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845042 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845111 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845156 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845200 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845251 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845315 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout = 60.0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845397 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845487 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845576 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845637 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type = password {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845688 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url = https://199.19.213.61/identity {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845732 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845801 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845850 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845906 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.845969 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846009 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846096 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type = public {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846144 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846190 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846279 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.password = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846353 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846397 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name = Default {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846453 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846656 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name = service {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846656 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name = RegionOne {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846656 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846656 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name = Default {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] nova.username = nova {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.846924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type = password {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type = public {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847698 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.847920 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848393 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848393 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848393 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848393 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848538 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ironic.version = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848538 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848599 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848599 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848663 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848866 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848902 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848949 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.848985 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849068 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000178251 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849233 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849310 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849349 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849444 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849520 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849600 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849635 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849665 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849697 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849765 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849824 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849922 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.849988 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850194 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850194 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850194 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850194 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850194 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850367 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850367 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850367 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850479 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850479 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_hostname_verification = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850556 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850556 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850621 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850682 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850741 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850787 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850862 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850927 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.850981 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851042 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851110 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851217 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota = -1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851289 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851355 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip = 50 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851409 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851501 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port = 500 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851617 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851617 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851751 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851801 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851887 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851943 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.851998 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852116 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852175 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852232 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852291 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852346 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852390 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852469 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852542 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852582 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.852867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853050 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853202 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853202 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853202 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853669 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853669 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853669 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853669 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853669 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities = [12, 21] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853830 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853830 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853830 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853830 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name = oslo_privsep.daemon {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853830 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size = 16 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853970 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853970 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.853970 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854039 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854098 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854135 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854184 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854228 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854315 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854396 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers = ['port_security', 'qos'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854454 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854516 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers = ['ovn'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854567 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version = 4 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854606 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854651 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus = {} {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854700 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types = ['geneve'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854734 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name = rp_tunnelled {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854784 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'geneve'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854847 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks = ['public'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854910 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.854964 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges = ['1:65536'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855036 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855108 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit = 25 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855152 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855265 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855315 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855367 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.backend = sqlalchemy {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855415 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.connection = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855475 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855518 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters = {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855572 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855614 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855656 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855699 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries = 20 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855734 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855782 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval = 1 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow = 50 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855940 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size = 5 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855940 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.855940 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode = TRADITIONAL {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856019 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856019 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout = None {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856166 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856211 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection = **** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856265 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856307 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856350 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856422 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx = 1000 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856481 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx = 100 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856518 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult = 3 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856573 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856615 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time = 43200 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856658 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856700 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856743 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers = [] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856784 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856820 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold = 300 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856866 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy = normal {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856909 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856952 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.856995 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold = 300 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857045 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode = log {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857090 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options = {} {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857154 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options = {} {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857226 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler = leastloaded {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857274 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857327 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857378 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857422 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection = ['ssl:199.19.213.61:6641'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857484 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857530 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857573 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857617 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857661 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection = ['ssl:199.19.213.61:6642'] {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857704 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857742 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857776 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout = 180 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857829 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level = INFO {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857860 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval = 60000 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857905 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval = 180 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857947 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.857996 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir = /var/run/openvswitch {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858057 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size = 50000 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858107 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858157 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports = True {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858222 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858256 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout = 10 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down = False {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3256}} Jun 03 01:14:23.858607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106388) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3263}} Jun 03 01:14:23.858801 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [106388] {{(pid=106388) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}} Jun 03 01:14:23.859559 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-1041295', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-3134860'] for process (None), after_init {{(pid=106388) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} Jun 03 01:14:23.861163 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry Jun 03 01:14:23.906992 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} Jun 03 01:14:23.907106 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} Jun 03 01:14:23.907106 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} Jun 03 01:14:23.907214 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907214 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907330 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907330 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.907695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.910824 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6641: connecting... Jun 03 01:14:23.917967 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6641: connected Jun 03 01:14:23.948745 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry Jun 03 01:14:23.981838 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} Jun 03 01:14:23.981838 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}} Jun 03 01:14:23.982012 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.982012 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=106388) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}} Jun 03 01:14:23.984207 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6642: connecting... Jun 03 01:14:23.989837 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6642: connected Jun 03 01:14:24.027951 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip precommit_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028098 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip precommit_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028185 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028185 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028306 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028368 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port precommit_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028611 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-2634551'] for port_forwarding_plugin (None), after_init {{(pid=106388) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} Jun 03 01:14:24.028710 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028777 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > router_interface after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028839 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.028947 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port_forwarding after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.029065 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > router_interface after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.029222 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension Jun 03 01:14:24.029313 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension Jun 03 01:14:24.029768 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment before_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.029897 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete_associations 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030285 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030383 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030480 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030561 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030644 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.030644 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.031225 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.031225 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.031225 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.090423 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.090607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.090607 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet before_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.090766 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet before_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.094108 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.094240 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.096122 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1359109'] for logging-plugin (None), after_init {{(pid=106388) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} Jun 03 01:14:24.096122 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_create 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.096122 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_update 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:24.096122 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > security_group_rule after_delete 55550000, False {{(pid=106388) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}} Jun 03 01:14:33.540750 np0000178251 neutron-ovn-maintenance-worker[105636]: DEBUG dbcounter [-] [105636] Writing DB stats neutron:SELECT=4 {{(pid=105636) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}} Jun 03 01:14:34.095957 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-06-03 01:14:34.095528 {{(pid=106388) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}} Jun 03 01:14:34.098874 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync port groups started @ 2026-06-03 01:14:34.098328 {{(pid=106388) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}} Jun 03 01:14:34.166601 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG dbcounter [-] [106388] Writer thread running {{(pid=106388) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}} Jun 03 01:14:34.183826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync port groups completed @ 2026-06-03 01:14:34.183514 {{(pid=106388) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}} Jun 03 01:14:34.183983 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-06-03 01:14:34.183809 {{(pid=106388) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}} Jun 03 01:14:34.507311 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.services.segments.db [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['9414260a-0dce-462c-b6b1-eaba1230a630'], 'is_dynamic': False}} {{(pid=106388) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}} Jun 03 01:14:34.513801 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync metadata ports started {{(pid=106388) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}} Jun 03 01:14:34.577540 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync metadata ports completed {{(pid=106388) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}} Jun 03 01:14:34.577807 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=106388) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}} Jun 03 01:14:34.641054 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-06-03 01:14:34.640790 {{(pid=106388) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}} Jun 03 01:14:34.642147 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} Jun 03 01:14:34.642348 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=106388) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}} Jun 03 01:14:34.642484 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=106388) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}} Jun 03 01:14:34.642547 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-06-03 01:14:34.642414 {{(pid=106388) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}} Jun 03 01:14:34.642691 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync ACLs started @ 2026-06-03 01:14:34.642550 {{(pid=106388) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}} Jun 03 01:14:34.686305 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync ACLs completed @ 2026-06-03 01:14:34.686000 {{(pid=106388) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}} Jun 03 01:14:34.686620 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync routers and router ports started @ 2026-06-03 01:14:34.686292 {{(pid=106388) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}} Jun 03 01:14:34.747352 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} Jun 03 01:14:34.747678 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync routers and router ports completed @ 2026-06-03 01:14:34.747356 {{(pid=106388) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}} Jun 03 01:14:34.747766 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync port QoS policies started @ 2026-06-03 01:14:34.747652 {{(pid=106388) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}} Jun 03 01:14:34.747886 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] Starting OVNClientQosExtension Jun 03 01:14:34.766204 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync port QoS policies completed @ 2026-06-03 01:14:34.766032 {{(pid=106388) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}} Jun 03 01:14:34.766396 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-06-03 01:14:34.766176 {{(pid=106388) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}} Jun 03 01:14:34.766396 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] Starting OVNClientQosExtension Jun 03 01:14:34.789517 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-06-03 01:14:34.789286 {{(pid=106388) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}} Jun 03 01:14:34.789686 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-06-03 01:14:34.789498 {{(pid=106388) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}} Jun 03 01:14:34.789846 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-06-03 01:14:34.789725 {{(pid=106388) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}} Jun 03 01:14:34.789936 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-7351bdf9-035e-4ecd-acc4-d362fe33ac36 None None] OVN-Northbound DB sync process completed @ 2026-06-03 01:14:34.789819 {{(pid=106388) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}} Jun 03 01:14:44.097583 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-06-03 01:14:44.096948 {{(pid=106388) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}} Jun 03 01:14:44.099592 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension Jun 03 01:14:44.101231 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] OVN-SB Sync hostname and physical networks started @ 2026-06-03 01:14:44.100605 {{(pid=106388) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}} Jun 03 01:14:44.105394 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.105394 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.106679 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.106883 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107013 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107081 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107162 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107228 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107409 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107507 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107573 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107652 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107716 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107780 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.107934 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108014 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108096 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108160 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108221 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108313 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108386 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108470 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108547 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108627 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.108696 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=106388) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}} Jun 03 01:14:44.110540 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.112067 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started Jun 03 01:14:44.112192 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization Jun 03 01:14:44.113074 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113528 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113528 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113528 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113528 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113528 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.113967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114209 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114209 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114209 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114411 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114411 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114590 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114662 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114728 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114793 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.114898 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.115027 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.115191 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.116196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.116277 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=106388) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}} Jun 03 01:14:44.117688 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Agents populated in the agent cache: {{(pid=106388) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:306}} Jun 03 01:14:44.117926 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] - Agent type: OVN Controller Gateway agent, hostname: np0000178251, name: 4fd7b9d9-c8d7-4cd3-a0c9-0b09c2ed4868 {{(pid=106388) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:308}} Jun 03 01:14:44.118095 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] - Agent type: OVN Neutron agent, hostname: np0000178251, name: 4fd7b9d9-c8d7-4cd3-a0c9-0b09c2ed4868 {{(pid=106388) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:308}} Jun 03 01:14:44.119385 np0000178251 neutron-ovn-maintenance-worker[106388]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] New host np0000178251 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron Jun 03 01:14:44.137994 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.138336 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.138609 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.197437 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.058 seconds Jun 03 01:14:44.197873 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.198024 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.198210 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.215046 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG oslo_concurrency.lockutils [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=106388) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:511}} Jun 03 01:14:44.215298 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG oslo_concurrency.lockutils [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=106388) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:524}} Jun 03 01:14:44.253799 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron_lib.callbacks.manager [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-6875259', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-532458'] for segment_host_mapping (None), after_create {{(pid=106388) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}} Jun 03 01:14:44.269744 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.services.segments.db [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Segments {'4477cce7-a42d-4b66-ae4b-14e1a6b0cf5b'} mapped to the host np0000178251 {{(pid=106388) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}} Jun 03 01:14:44.276185 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG oslo_concurrency.lockutils [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.061s {{(pid=106388) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:543}} Jun 03 01:14:44.276314 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] OVN-SB Sync hostname and physical networks completed @ 2026-06-03 01:14:44.276175 {{(pid=106388) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}} Jun 03 01:14:44.281972 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.083 seconds Jun 03 01:14:44.282201 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.282363 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.282514 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.282841 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds Jun 03 01:14:44.282931 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.283123 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.283208 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.293363 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:44.388216 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.388663 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.391668 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.108 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.408098 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.408479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.408860 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.408860 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.411825 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.412329 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.412898 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds Jun 03 01:14:44.413091 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.413091 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.413304 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.435778 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=106388) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}} Jun 03 01:14:44.435933 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_for_inconsistencies finished in 0.023 seconds Jun 03 01:14:44.436661 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.436661 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.436661 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds Jun 03 01:14:44.436919 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.437113 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.437203 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.438129 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} Jun 03 01:14:44.438401 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.001 seconds Jun 03 01:14:44.438585 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.438724 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.438853 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.563270 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.124 seconds Jun 03 01:14:44.565728 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.565997 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.566366 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.774361 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_provider_distributed_ports finished in 0.208 seconds Jun 03 01:14:44.774570 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.774824 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.774919 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.905964 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} Jun 03 01:14:44.906550 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-df75013c-795b-4e34-83cf-1441d06e027e None None] OVN-Southbound DB sync process completed @ 2026-06-03 01:14:44.906178 {{(pid=106388) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}} Jun 03 01:14:44.921434 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.146 seconds Jun 03 01:14:44.921616 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.921961 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.922149 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.923744 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.924051 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.924799 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.925686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.926248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.926889 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.927585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.928396 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.931893 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.db.ovn_hash_ring_db [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Cleaned up Hash Ring nodes older than 5 days Jun 03 01:14:44.932031 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.005 seconds Jun 03 01:14:44.932292 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: configure_nb_global {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.932409 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.932744 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:44.933338 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.933944 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.934770 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:44.935843 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.936325 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task configure_nb_global failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.937034 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.937380 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.937846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.937846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.937846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.937846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.938294 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.938775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}} Jun 03 01:14:44.939311 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.939541 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.939654 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.939927 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:14:44.940531 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.940870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.941780 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:44.942671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.943046 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.943666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.944013 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.944365 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.944365 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.944365 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.944365 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.944764 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.945186 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.945676 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.954498 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_network_type_and_physnet finished in 0.009 seconds Jun 03 01:14:44.954648 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.954817 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.954983 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.955270 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds Jun 03 01:14:44.955949 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.955949 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.955949 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.956155 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:44.956588 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.956994 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.957853 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:44.958748 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.959394 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.959787 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.960275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.960857 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.960857 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.960857 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.960857 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.961032 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.961806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.001 seconds Jun 03 01:14:44.962619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.963015 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.963015 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.963015 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:44.963625 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.964049 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:44.964270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.965085 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.966174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.966523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:44.966858 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.966858 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:44.967289 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:44.967725 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:44.968358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:44.968358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:44.968358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:44.968358 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.968358 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.975013 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_router_distributed_flag finished in 0.007 seconds Jun 03 01:14:44.975185 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.975359 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:44.975532 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:44.987395 np0000178251 neutron-ovn-maintenance-worker[106388]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_security_group_with_address_group finished in 0.012 seconds Jun 03 01:14:44.987527 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:44.988342 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106388) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}} Jun 03 01:14:49.283835 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:49.284880 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:49.284880 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:49.286556 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:49.320568 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.320907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.321580 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.322668 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.323426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:49.324093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:49.324911 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.325863 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:49.922583 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:49.923769 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:49.923972 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:49.925168 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.925621 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.926606 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.927689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.928120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:49.928673 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.928673 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:49.929218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:49.929976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.931049 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:49.931049 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.931049 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:49.939824 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:49.940278 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:49.940376 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:49.940498 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:14:49.941090 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.942231 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:49.942943 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:49.943726 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.944344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.945002 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:49.945648 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.946759 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.947711 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:49.956064 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:49.956467 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:49.956601 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:49.956744 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:49.957352 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.957802 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.958737 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:49.960028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.960499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.961539 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:49.962216 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:49.963073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.963073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:49.963073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.963073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.963392 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:49.964615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:49.965636 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:49.966083 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.966083 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:49.966083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.967869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.968484 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:49.968892 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:49.969296 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:49.969981 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:54.284695 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:54.285279 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:54.285348 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:54.287164 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:54.326249 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.326874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.328737 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.329586 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.330166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:54.330755 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:54.332027 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.333468 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:54.923774 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:54.924619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:54.924619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:54.925352 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.925750 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.926603 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.928814 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.929432 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:54.930150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:54.931418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.932535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:54.940385 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:54.940827 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:54.941018 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:54.941251 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:14:54.941851 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.942178 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.943057 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:54.944426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.944923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.945855 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:54.946231 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:54.946639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.946639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:54.946639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.946639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:54.947438 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.947942 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.948427 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:54.956666 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:54.956986 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:54.957103 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:54.957233 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:54.957642 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.957923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.958587 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:54.959302 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.959839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.960222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:54.960615 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:54.961008 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.961008 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:54.961008 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.961008 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.962070 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:54.962705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:54.963259 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.963259 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:54.963259 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.963259 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:54.964190 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:54.964636 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:54.964820 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:54.965073 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:54.965459 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.965854 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:54.966053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.966880 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.968341 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.968747 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:54.969168 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.969168 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:54.969330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:54.969870 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:54.970481 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:54.970481 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:54.970481 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:59.285303 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:59.286293 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:59.286293 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:59.287416 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:59.322936 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.323363 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.324282 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.325053 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.325881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:59.326695 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:59.327821 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.329527 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:59.924616 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:59.925043 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:59.925233 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:59.926455 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.926848 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.927875 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.929702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.930045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:59.930359 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.930359 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:59.931656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:59.932205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.932957 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:59.932957 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.932957 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:59.940928 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:59.941394 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:59.941616 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:59.941882 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:14:59.942598 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.943035 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.944006 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:59.945416 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.946251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.947056 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:59.948073 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:59.948833 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.948833 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:59.948833 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.948833 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:59.949072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.949865 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.951140 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:59.957163 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:59.957508 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:59.957638 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:59.957918 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:59.958533 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.959108 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.959997 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:14:59.961399 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.962201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.963067 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:59.964115 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:59.964803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.964803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:59.964803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.964803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.965165 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:59.966019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:14:59.966968 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:14:59.967405 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.967405 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:14:59.967405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.968213 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.969190 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.969608 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:14:59.970018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.970018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:14:59.970270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:14:59.970812 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:14:59.971397 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:14:59.971397 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:14:59.971397 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:04.286027 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:04.286772 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:04.287010 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:04.288493 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:04.322057 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.322546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.323411 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.324646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.325275 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:04.326089 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:04.326835 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.327839 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:04.927794 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:04.928934 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:04.928934 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:04.929967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.930298 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.931169 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.932524 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.933507 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:04.934261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:04.935149 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.936263 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:04.941566 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:04.942040 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:04.942244 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:04.942381 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:04.943104 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.943499 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.945017 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:04.945671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.946151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.946702 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:04.947081 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:04.947511 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.947511 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:04.947511 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.947511 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:04.948335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.948869 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.949344 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:04.957730 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:04.959400 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:04.959400 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:04.959400 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:04.959856 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.960125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.961075 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:04.962054 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.963172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.963853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:04.964647 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:04.965736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:04.966291 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:04.967071 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:04.967613 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:04.967613 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:04.967613 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.967613 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:04.967613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.967840 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.968638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.969182 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:04.969694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.969694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:04.969784 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:04.970389 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:04.971129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:04.971129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:04.971129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:09.286869 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:09.287880 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:09.287880 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:09.288819 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:09.330006 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.330476 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.331535 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.332390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.333052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:09.333700 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:09.334405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.335387 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:09.928687 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:09.929022 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:09.929203 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:09.930425 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.930858 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.931952 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.933546 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.934174 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:09.934803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.934803 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:09.935542 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:09.936338 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.937271 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:09.937271 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.937271 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:09.942062 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:09.942342 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:09.942482 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:09.942697 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:09.943281 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.943701 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.944703 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:09.946261 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.946751 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.947736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:09.948145 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:09.948584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.948584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:09.948584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.948584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:09.948984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.949429 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.949986 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:09.958868 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:09.959090 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:09.959204 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:09.959398 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:09.960183 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.960585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.961273 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:09.962143 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.962564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.963152 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:09.963506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:09.963876 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.963876 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:09.963876 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.963876 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.964426 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:09.964919 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:09.965403 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.965403 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:09.965403 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.965403 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:09.966212 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:09.966455 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:09.966565 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:09.966746 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:09.967309 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.967707 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:09.967847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.968523 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.969462 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.969822 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:09.970205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.970205 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:09.970565 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:09.971018 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:09.971584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:09.971584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:09.971584 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:14.287653 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:14.290472 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:14.290472 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:14.290472 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:14.526750 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.527388 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.528489 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.240 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.529614 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.530283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:14.530932 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:14.531652 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.532576 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:14.929654 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:14.931515 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:14.931515 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:14.931941 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.932151 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:14.935646 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:14.936265 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.936692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.937102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.937729 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:14.942642 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:14.942983 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:14.943196 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:14.943413 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:14.944199 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.944466 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.948068 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:14.948710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.949160 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.949691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:14.950225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:14.950679 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.951229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:14.959345 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:14.959676 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:14.959881 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:14.960099 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:14.960834 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.961207 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.964660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:14.965253 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.965694 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.966080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:14.966422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:14.966856 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:14.967382 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:14.967908 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:14.968111 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:14.968349 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:14.968917 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.969220 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:14.969405 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:14.971357 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:14.971691 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.971959 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:14.972238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:14.972624 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:19.290069 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:19.294466 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:19.294466 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:19.294466 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:19.335357 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.335705 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.336568 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.339467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.340031 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:19.340656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.340656 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:19.341420 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:19.342170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.342989 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:19.342989 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.342989 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:19.930633 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:19.931424 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:19.932537 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:19.933270 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.933740 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:19.936947 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:19.937575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.938407 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.939062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.939728 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:19.943127 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:19.943473 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:19.943651 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:19.943870 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:19.944451 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.944840 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.945572 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:19.948303 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:19.948987 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.949655 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.950201 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:19.950761 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.951651 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.952969 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:19.959890 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:19.960674 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:19.960928 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:19.961246 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:19.961937 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.962307 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.966120 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:19.967248 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.968530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.969179 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:19.969504 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:19.969799 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:19.970162 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:19.970643 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:19.970800 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:19.970975 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:19.971550 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.971904 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:19.972117 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:19.974371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:19.974685 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.974954 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:19.975217 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:19.975601 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:24.291763 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:24.294003 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:24.294003 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:24.294003 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.334837 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.335599 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.335887 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:24.336127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.336127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:24.337041 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:24.337395 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.337853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:24.337853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.337853 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:24.931652 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:24.933173 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:24.933173 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:24.934065 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.934424 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:24.937752 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:24.938238 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.938677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.939074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.939660 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:24.943833 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:24.944209 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:24.944424 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:24.944708 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:24.945308 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.945561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.949304 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:24.949846 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.950326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.950778 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:24.951164 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:24.951609 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.952157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:24.960960 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:24.961732 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:24.961961 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:24.962341 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:24.963262 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.963831 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.967477 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:24.968072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.968568 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.968902 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:24.969270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.969270 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.970502 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:24.978639 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:24.979136 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:24.979381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.979781 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.980157 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:24.980582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:24.981019 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:24.981706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:24.981706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:24.981706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:29.292951 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:29.293691 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:29.293691 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:29.294967 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:29.329164 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.329471 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.330003 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.330723 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.331650 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:29.332501 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:29.333676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.334930 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:29.932506 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:29.933163 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:29.933344 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:29.934869 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.935252 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.936400 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.938356 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.939276 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:29.939831 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.939831 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:29.940605 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:29.941127 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.941706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:29.941706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.941706 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:29.944304 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:29.944767 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:29.944867 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:29.944997 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:29.945831 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.946204 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.947211 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:29.948147 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.948880 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.949364 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:29.949862 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:29.950564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.950564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:29.950564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.950564 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:29.950776 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.951328 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.951903 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:29.961756 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:29.962465 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:29.962696 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:29.962922 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:29.964033 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.964556 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.965620 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:29.966970 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.967497 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.968222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:29.968622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:29.969017 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.969017 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:29.969017 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.969017 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.969400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:29.969953 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:29.970561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.970561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:29.970561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.970561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:29.971551 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:29.971895 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:29.972006 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:29.972182 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:29.972805 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.973088 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:29.973211 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.973832 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.974785 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.975251 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:29.975818 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.975818 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:29.975906 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:29.976547 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:29.977254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:29.977254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:29.977254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:34.293919 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:34.294733 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:34.294733 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:34.296111 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:34.298145 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG dbcounter [-] [106388] Writing DB stats neutron:SELECT=520,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=106388) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}} Jun 03 01:15:34.330888 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.331218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.332204 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.333467 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.334052 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:34.334642 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:34.335292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.336237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:34.933251 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:34.934099 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:34.934389 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:34.935388 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.935881 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.936910 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.938938 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.939575 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:34.940195 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.940195 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:34.941169 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:34.941907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.942425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:34.942425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.942425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:34.944945 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:34.945352 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:34.945576 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:34.945780 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:34.946378 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.946810 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.947793 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:34.953613 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:34.954292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.954937 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.955509 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:34.956062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.956955 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.957842 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:34.962432 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:34.963022 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:34.963240 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:34.963565 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:34.963969 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.964218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.964888 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:34.966521 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.967229 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.968209 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:34.968783 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:34.969492 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.969492 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:34.969492 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.969492 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.970102 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:34.971240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:34.972172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.972172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:34.972172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.972172 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:34.972172 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:34.972590 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:34.972843 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:34.973128 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:34.973540 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.973726 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:34.973806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.974415 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.976093 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.976425 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:34.976780 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.976780 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:34.977967 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:34.978401 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:34.978922 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:34.978922 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:34.978922 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:39.294697 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:39.295697 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:39.295697 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:39.296742 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:39.333888 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.334337 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.335162 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.335847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.336590 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:39.337278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:39.338083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.339129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:39.934205 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:39.934702 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:39.934893 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:39.945407 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:39.947676 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.947676 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.948734 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.014 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.950696 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.951692 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:39.952659 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.952659 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:39.953326 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:39.954791 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.955471 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.956036 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:39.956979 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.957672 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.958223 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:39.958770 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:39.959322 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.959322 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:39.959322 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.959322 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:39.959760 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.960246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.960793 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:39.962833 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:39.963123 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:39.963263 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:39.963461 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:39.964111 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.964596 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.965465 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:39.966381 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.967004 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.967665 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:39.968233 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:39.968844 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.968844 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:39.968844 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.968844 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.969222 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:39.969772 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:39.970292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.970292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:39.970292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.970292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:39.972609 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:39.972876 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:39.973042 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:39.973251 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:39.973831 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.974327 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:39.974645 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.975561 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.976641 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.977047 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:39.977374 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.977374 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:39.977666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:39.978324 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:39.979130 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:39.979130 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:39.979130 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:44.295396 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:44.296195 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:44.296433 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:44.298057 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:44.332371 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.332649 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.333278 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.334418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.335001 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:44.335645 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.335645 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:44.335749 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:44.336482 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.337258 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:44.337258 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.337258 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:44.935044 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:44.935401 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:44.935653 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:44.936766 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.937166 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.938136 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.939721 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.940125 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:44.940508 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.940508 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:44.941538 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:44.942006 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.942560 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:44.942560 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.942560 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:44.953736 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:44.954045 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:44.954224 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:44.954413 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:44.954954 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.955309 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.956189 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:44.957351 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.958256 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.958686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:44.959202 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:44.959913 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.959913 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:44.959913 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.959913 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:44.960278 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.961109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.961991 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:44.963657 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:44.963924 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:44.964031 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:44.964159 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:44.964703 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.965087 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.965974 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:44.966766 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.967107 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.967680 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:44.967964 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:44.968257 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.968257 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:44.968257 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.968257 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.968820 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:44.969170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:44.969571 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.969571 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:44.969571 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.969571 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:44.972955 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:44.973148 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:44.973250 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:44.973401 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:44.973912 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.974287 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:44.974520 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.975241 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.976150 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.976566 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:44.976933 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.976933 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:44.977815 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:44.978335 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:44.979074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:44.979074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:44.979074 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:49.296405 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:49.297542 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:49.297542 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:49.298676 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:49.334874 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.335273 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.336240 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.337345 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.337745 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:49.338091 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.338091 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:49.338400 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:49.338874 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.339383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:49.339383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.339383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:49.935738 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:49.936491 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:49.936688 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:49.937885 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.938246 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.939218 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.940878 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.941518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:49.942103 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.942103 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:49.942976 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:49.943756 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.944710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:49.944710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.944710 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:49.954145 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:49.954566 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:49.954758 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:49.954951 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:49.955555 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.955907 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.956813 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:49.958138 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.958909 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.959585 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:49.960300 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:49.961148 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.961900 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.962899 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:49.964151 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:49.964463 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:49.964588 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:49.964717 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:49.965211 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.965654 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.966388 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:49.967184 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.967730 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.968133 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:49.968518 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:49.968936 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.968936 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:49.968936 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.968936 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.969264 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:49.969794 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:49.970358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.970358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:49.970358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.970358 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:49.973325 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:49.973621 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:49.973728 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:49.973875 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:49.974357 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.974780 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:49.974984 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.975775 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.976877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.977225 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:49.977602 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.977602 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:49.978390 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:49.978847 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:49.979373 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:49.979373 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:49.979373 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:54.297687 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:54.298556 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:54.298640 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:54.299985 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:54.342532 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.343062 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.344301 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.345371 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.346506 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:54.347580 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:54.348806 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.350418 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:54.939558 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:54.939939 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:54.940063 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:54.940970 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.941232 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.942079 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.943786 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.944583 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:54.945207 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.945207 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:54.945638 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:54.946330 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.947173 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:54.947173 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.947173 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:54.954650 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:54.954864 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:54.954986 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:54.955109 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:54.955717 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.956039 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.956809 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:54.958158 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.959170 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.959735 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:54.960343 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:54.961080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.961080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:54.961080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.961080 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:54.961675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.962530 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.963516 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:54.964619 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:54.965006 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:54.965179 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:54.965436 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:54.966133 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.966582 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.967283 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:54.968078 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.968677 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.969092 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:54.969683 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:54.970177 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:54.970836 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.971622 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:54.973754 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:54.974054 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:54.974166 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:54.974310 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:54.974844 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.975188 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:54.975367 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.976060 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.976971 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.977512 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:54.978109 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:54.978743 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:54.979555 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:59.298661 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:59.299741 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:59.299741 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:59.301493 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:59.342939 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.343675 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.344598 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.345479 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.346292 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:59.347045 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:59.348686 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.350072 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:59.949851 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:59.953677 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:59.954019 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:59.956601 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:59.957160 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-e7a97bd0-ffce-4b76-a097-bc02bf0343c0, routes=[], if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.957637 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.959237 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.960333 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.961025 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:59.961681 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.961681 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:59.962266 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:59.963949 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106388) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1163}} Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.965826 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.966254 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:59.967834 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^ Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.968877 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.969129 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:59.969383 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1167, in set_fip_distributed_flag Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute( Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:59.969689 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:59.970060 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:59.970309 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last): Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command next(self.gen) Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command with self.create_transaction( Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command self.result = self.commit() Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^ Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise result.ex Jun 03 01:15:59.970671 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command txn.results.put(txn.do_commit()) Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command ^^^^^^^^^^^^^^^ Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command raise RuntimeError(msg) Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.command Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:59.970988 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:59.971255 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.971574 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.971939 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1228, in update_ha_failover Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self._nb_idl.set_nb_global_options(**options).execute( Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.api.transaction(check_error, log_errors, **kwargs) as t: Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:59.972736 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:59.973607 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.973607 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:59.973607 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.973607 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:15:59.974183 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:15:59.974601 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:15:59.974601 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:15:59.974742 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:15:59.975066 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.975338 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:15:59.975519 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.976000 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.976921 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.977788 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:15:59.978315 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.978315 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:15:59.978480 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:15:59.978923 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:15:59.979422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:15:59.979422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:15:59.979422 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Jun 03 01:16:04.299434 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106388) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}} Jun 03 01:16:04.300232 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}} Jun 03 01:16:04.300496 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106388) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}} Jun 03 01:16:04.302483 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106388) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:963}} Jun 03 01:16:04.339955 np0000178251 neutron-ovn-maintenance-worker[106388]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106388) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Jun 03 01:16:04.340105 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Traceback (most recent call last): Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: txn.results.put(txn.do_commit()) Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: ^^^^^^^^^^^^^^^ Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: raise RuntimeError(msg) Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.341015 np0000178251 neutron-ovn-maintenance-worker[106388]: Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last): Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance result = f(*args, **kwargs) Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^^^^ Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with super().transaction(*args, **kwargs) as t: Jun 03 01:16:04.342083 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance next(self.gen) Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance with self.create_transaction( Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance self.result = self.commit() Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^ Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise result.ex Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance txn.results.put(txn.do_commit()) Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ^^^^^^^^^^^^^^^ Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:16:04.343028 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance raise RuntimeError(msg) Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics [None req-ee11573c-c32e-4f72-b588-f8218a5b13f6 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics Traceback (most recent call last): Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics work() Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__ Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return self.callback(*self.args, **self.kwargs) Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(*args, **kwargs) Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics return f(self, *args, **kwargs) Jun 03 01:16:04.343868 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^^^^^^^ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics result = f(*args, **kwargs) Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^^^^ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self._nb_idl.transaction(check_error=True) as txn: Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 273, in transaction Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with super().transaction(*args, **kwargs) as t: Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics next(self.gen) Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics with self.create_transaction( Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics self.result = self.commit() Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^ Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise result.ex Jun 03 01:16:04.344722 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics txn.results.put(txn.do_commit()) Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics ^^^^^^^^^^^^^^^ Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics raise RuntimeError(msg) Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it Jun 03 01:16:04.345666 np0000178251 neutron-ovn-maintenance-worker[106388]: ERROR futurist.periodics